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ABSTRACT 

Introduction: This project concerns the identification 
of the smallest worthwhile effect (SWE) of 
exercise-based programmes to prevent falls in older 
people. The SWE is the smallest effect that 
justifies the costs, risks and inconveniences of an 
intervention and is used to inform the design and 
interpretation of systematic reviews and randomised 
clinical trials. 

Methods and analysis: This study will comprise 
two different methodological approaches: the benefit- 
harm trade-off method and the discrete choice 
experiment to estimate the SWE of exercise 
interventions to prevent falls in older people. In the 
benefit-harm trade-off method, hypothetical scenarios 
with the benefits, costs, risks and inconveniences 
associated with the intervention will be presented to 
each participant. Then, assuming a treatment effect of 
certain magnitude, the participant will be asked if he 
or she would choose to have the intervention. The 
size of the hypothetical benefit will be varied up and 
down until it is possible to identify the SWE for 
which the participant would choose to have the 
intervention. In the discrete choice experiment, the 
same attributes (benefits, costs, risks and 
inconveniences) with varying levels will be presented 
as choice sets, and participants will be asked to 
choose between these choice sets. With this 
approach, we will determine the probability that a 
person will consider the effects of an intervention to 
be worthwhile, given the particular costs, risks and 
inconveniences. For each of the two approaches, 
participants will be interviewed in person and on 
different occasions. A subsample of the total cohort 
will participate in both interviews. 
Ethics and dissemination: This project has 
received Ethics Approval from the University of 
Sydney Human Ethics Committee (Protocol number: 
14404). Findings will be disseminated through 
conference presentations, seminars and peer-reviewed 
scientific journals. 



ARTICLE SUMMARY 



Article focus 

■ To determine the smallest worthwhile effect 
(SWE) of exercise programmes to prevent falls 
and investigate the relative importance of various 
factors that influence older people's decision to 
participate or not in these programmes. 

■ To determine the extent of trade-offs between 
harms and benefits that older people are willing 
to accept in making decisions about participation 
in exercise programmes to prevent falls. 

■ To investigate whether the benefit-harm trade-off 
method and the discrete choice experiment yield 
similar estimates of the SWE of an exercise pro- 
gramme to prevent falls. 

Key messages 

■ Despite the clear evidence that the rate of falls in 
older people can be reduced with exercise inter- 
ventions, there has not yet been any formal 
evaluation of whether potential recipients of such 
interventions consider effects of the magnitude 
observed in randomised trials to be worthwhile 
to justify the costs and inconveniences they 
experience in participating. 

■ Trade-offs between the potential benefits and 
harms of exercise programmes to prevent falls 
should be weighed by older people deciding 
whether to participate or not in these exercise 
programmes. 

■ The findings of this study should enable the con- 
struction of fall prevention programmes with 
high participation rates, high levels of adherence 
and high levels of perceived benefit. 



INTRODUCTION 

Randomised controlled trials have become the 
method of choice for determining the effects 
of health interventions. More than 500 000 
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Strengths and limitations of this study 

■ This is the first study to use robust stated-preference methods 
(discrete choice methods and benefit-harm trade-off method) 
to estimate the smallest worthwhile effect of exercise-based 
programs to prevent falls. 

■ The results will inform clinical practice, research and policy, as 
the attributes of an exercise to prevent falls most valued by 
consumers will be identified. 

■ The estimates of the smallest worthwhile effect should be 
interpreted in the context of the included population and inter- 
ventions only (ie, exercise programs to prevent falls in older 
people compared to no treatment). 

randomised trials have been conducted in health interven- 
tions, almost all in the last 50 years. There is a high degree 
of consensus about how randomised trials should be con- 
ducted. 1 Nonetheless, several important methodological 
issues remain unresolved. One of the most persistent 
issues concerns how to estimate the smallest worthwhile 
effect (SWE) of an intervention. 

The SWE is, as its name suggests, the smallest benefi- 
cial effect of an intervention that justifies the costs, risks 
and inconvenience of that intervention. Estimates of the 
SWE of an intervention are needed to design powerful 
and efficient clinical trials and to determine whether 
interventions produce effects that are large enough to 
be worthwhile. 2 Several approaches have been used to 
estimate the SWE of intervention. These have been crit- 
ically reviewed by Barrett et at and Ferreira et al. 4 The 
authors argue that any valid measure of the SWE of 
intervention must have three characteristics: judgements 
about whether the effects of intervention are large 
enough to be worthwhile must be made by recipients of 
care, the estimate must be intervention-specific and the 
method must focus on the effects of intervention 
(between-group differences) rather than on changes 
over time (within-group differences) . 

Many randomised clinical trials and systematic reviews 
have investigated the impact of interventions targeted at 
preventing falls in older people and now provide clear 
evidence that both the rate of falls and risk of falling in 
older people can be reduced with exercise interventions, 
which for instance involve challenge to balance abilities 
and are performed frequently (ie, at multiple times a 
week over a long period) . 5 6 However, there has not yet 
been any formal evaluation of whether potential recipi- 
ents of such interventions consider effects of the magni- 
tude observed in randomised trials to be worthwhile to 
justify the costs and inconveniences they experience in 
participating. 

This paper presents the protocol of a study designed 
to determine the SWE of a fall-prevention programme. 
The study will use a modified contingent valuation 
method (the 'benefit-harm trade-off method') and the 
discrete choice experiment to determine what older 
people who have previously fallen consider to be the 



SWE of exercise to prevent falls from their own 
perspective. 

The aims of the study are to answer the following 
questions: 

1 . What do older people who have previously fallen con- 
sider to be the SWE of an exercise-based fall preven- 
tion programme? 

2. To what extent is the SWE of an exercise-based fall 
prevention programme influenced by expectations of 
the costs, risks or inconveniences of intervention? 

3. What characteristics of exercise-based fall prevention 
programmes do older people who have previously 
fallen value most? 

4. Do the benefit-harm trade-off method and discrete 
choice experiment yield similar estimates of the SWE 
of an exercise-based fall prevention programme? 

METHODS AND ANALYSIS 
Overview of approach and methods 

Different face-to-face interviews will be conducted for 
each experiment. The first interview concerns the 
benefit-harm trade-off method and will be used to 
address aim 1. The second interview will contribute to 
the discrete choice experiment and will address aims 
1-3. A subsample of 60 participants will participate in 
both interviews to address aim 4. A systematic review of 
qualitative studies will inform the development of both 
designs. The review seeks information on the percep- 
tions and experiences of older people on barriers and 
facilitators of exercise programmes to prevent falls. Data 
from this qualitative review will be used to determine 
the potential attributes for both the benefit-harm trade- 
off and the discrete choice experiment, such as benefits, 
costs and inconveniences associated with an exercise- 
based fall prevention programme. 

In the benefit-harm trade-off method, these attributes 
will have fixed levels and participants will decide the 
smallest expected benefit of intervention for which he 
or she would consider choosing to have the interven- 
tion. In contrast, in the discrete choice experiment, par- 
ticipants are asked to choose between alternatives 
defined by a set of attributes with varying levels. In this 
case, we can identify which attributes are driving partici- 
pants' preferences, the trade-offs participants make 
between attributes and how changes in attributes can 
lead to changes in the patients' willingness to participate 
in the intervention. 

Participants 

Participants will be recruited via newspaper, radio and 
online media advertisements as well as through commu- 
nity organisations that target older audiences. To be eli- 
gible to participate in both the benefit-harm trade-off 
study and the discrete choice experiment, study partici- 
pants must meet the criteria below: 

► Community-dwelling people aged 60 years or over, 

► Able to comprehend and read English fluently, 
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► Have experienced one or more falls in the past, 

► Present no obvious cognitive impairment, 

► Present no serious neurological, cardiovascular or 
musculoskeletal condition that might hinder their 
participation in exercise programprogrammes. 

Participants will not be excluded based on their par- 
ticipation in exercise programmes (ie, participants who 
are participating and who are not participating in exer- 
cise programmes will be eligible). 

Benefit-harm trade-off method 

For this approach, the interviewer will describe, for each 
participant, the potential benefits as well as the costs 
and risks and inconveniences associated with an 
exercise-based programme to prevent falls. The partici- 
pant will then be told to assume that the treatment 
effect will be of a certain size. Participants will then be 
asked if, given this expected effect of treatment, they 
would choose to have the intervention. Subsequently, 
estimates of the SWE will be elicited by asking the par- 
ticipant to consider the situation in which, hypothetic- 
ally, the expected effect of intervention is larger or 
smaller. The size of the hypothetical benefit will be 
varied up and down in progressively smaller increments 
until it is possible to identify the threshold benefit of 
intervention for which the participant would consider 
choosing to have the intervention. This is the SWE for 
that participant. 

Discrete choice experiment 

The discrete choice experiment will be conducted in a 
way that is consistent with current recommendations. 7 
This part of the study will use a survey which includes 
the same attributes associated with an exercise-based 



programme to prevent falls. Participants will be pre- 
sented with multiple choice sets of two hypothetical pro- 
grammes, where the levels of each attribute vary 
systematically between alternatives and scenarios, and 
will choose the optimal alternative in each choice set. As 
it is not feasible to present all participants with all pos- 
sible combinations of attribute levels, each participant 
will be presented with a subset of all possible choice sets. 
An efficient design will be used. 8 With this approach, 
the aim is to present participant choice sets that minim- 
ise the elements of the asymptotic variance-covariance 
matrix of the statistical methods used to analyse the 
data, that is, the aim is to maximise the precision of esti- 
mates of the value of attributes. 

From participants' choices, a mathematical function 
that describes numerically the value that respondents 
attach to different choice options will be estimated. This 
is one way to quantify patients' preferences for healthcare 
programmes. The results will be used to estimate the rela- 
tive value attached to attributes by examining trade-offs 
that people are willing to make between attributes. As a 
consequence, it is possible to determine the probability 
that a person will consider the effects of an exercise- 
based programme to prevent falls worthwhile, given the 
particular costs, risks and inconveniences. 

Table 1 is an example of a discrete choice study. 
The example includes nine attributes (costs, transpor- 
tation alternatives, travel time, type of exercise, fre- 
quency, time per day, improvement in the ability to 
undertake daily tasks at home, improvement in the 
ability to leave the house to undertake tasks and to 
socialise and falls risk reduction), with their specific 
levels. Participants would be asked to choose pro- 
gramme 1 or 2. 



Table 1 An example of a discrete choice study 










Exercise 


Scenario 1 


Exercise option A 


option B 


Out of pocket cost ($ per session) 


$100 


$5 


Is transport provided 


No transport is provided; you need to 


No need for 




provide your own 


transport 


Travel time 


About 45 min 


5 min or less 


Type of exercise 


Combination of different types of 


Yoga 




exercise (balance and strength training), 




How often do you exercise per week? 


1 x/week 


1 x/week 


Time per day 


30 min 


10 min or less 


Improvement in the ease with which you can undertake daily 


10% 


30% 


tasks at home (daily tasks at home include bathing, dressing, 






preparing meals and cleaning the house) 






Improvement in the ability to leave the house to undertake tasks 


30% 


60% 


or socialise (Tasks include shopping, banking, walking outdoors, 






using public transport) 






Falls risk reduction 


20 of 100 


10 of 100 


On average, 30 of 100 older people fall at least once each year. 






Exercise can reduce the number of people who fall to... 






p!^^""^ Which would you choose? DPrefer option A DPrefer option B. 
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The initial design will be tested with a pilot study 
including community-dwelling older people living in 
Australia to assess comprehension and understanding of 
attributes and their levels. From the findings of this pilot 
study, we will be able to estimate prior parameters, 
which will inform the efficient design of the final study. 
Strategies to facilitate appropriate interpretations of the 
DCE questionnaire will be used, such as diagrams and 
plain language. If the final design results in a large 
number of scenarios, the design will be blocked into two 
blocks to ensure that respondents are not overly bur- 
dened by the number of questions. 

Sample size 

For the benefit-harm trade-off experiment, our simula- 
tions indicate that, at least with normally distributed 
data, a sample size of 60 participants will provide 
expected CI widths of ±<0.4 SDs. 

For the discrete choice experiment, the current 
theory of sampling determines that sample sizes are 
based upon the characteristics of the design of the study, 
such as the number of attributes included, the attribute 
level range, the number of hypothetical scenarios pre- 
sented and the number of alternatives in each choice 
set. 9 Consequently, the sample size cannot be deter- 
mined until the attributes to be included in the final 
design are identified. An efficient design, which mini- 
mises D-error, will be estimated using NGENE software. 
D-error is a measure of statistical efficiency of the design 
(lower D-error indicates greater design efficiency). With 
an efficient choice design, we expect that a sample size 
of approximately 200 respondents will be sufficient to 
answer our questions of interest. 

Data analysis 

Benefit-harm trade-off method 

The distributions of estimates of the SWE elicited in 
the benefit-harm trade-off will be plotted as frequency 
histograms. Non-parametric bootstrap methods 10 will be 
used to generate 95% CIs for effects that are considered 
to be worthwhile by 20, 50 and 80% of participants. 11 

Discrete choice experiment 

A summary of descriptive statistics will be calculated for 
respondent samples. Data will be analysed with a mixed 
multinomial logit model (MMNL) . With this approach, it is 
implicitly assumed that preferences do not vary between 
responses made by an individual respondent, but variation 
in preferences between respondents is modelled explicitly. 
Mixed models allow for dependence of observations pro- 
vided by the same respondent. 12-14 The use of MMNLs 
relaxes the statistical assumptions made with more com- 
monly used fixed effect multinomial logit models and is 
likely to better explain choice behaviour than fixed effect 
models. 15 16 Interactions between attributes and between 
attributes and population characteristics (eg, age, gender, 
education and prior recent experience with falls) will be 
explored by including the appropriate interaction terms in 



the model. The analysis willprovide an estimate of the prob- 
ability that participants consider an intervention to be 
worthwhile (or, more directly, of the log odds that partici- 
pants would choose to have the intervention) based on any 
particular combination of attributes, and also will allow us 
to examine the trade-offs between attributes that partici- 
pants are willing to accept. In accordance with the recom- 
mendations of Lancsar and Louviere, responses deemed by 
researchers as 'irrational' will not be excluded from analysis, 
and all available responses will be included in the model. 17 



DISCUSSION 

This project will determine, from the older person's per- 
spective, the SWE of an exercise programme designed to 
prevent falls. Estimates of the SWE can be used to deter- 
mine whether the effects of new and existing fall preven- 
tion programmes are large enough to justify their 
implementation. 

The outcomes will be important for falls researchers 
and policy makers because they will provide the first 
rigorous analysis of the features of falls prevention pro- 
grammes that might increase the participants' participa- 
tion and adherence to these programmes. Therefore, 
the results could be used to optimise interventions, as 
they will identify attributes of falls prevention pro- 
gramme that maximise value to the recipients. 

This project also has a broader significance as it involves 
the development of novel methods that are applicable to 
the design and interpretation of clinical trials across all 
areas of healthcare. Benefit-harm trade-off studies are 
simple enough to be routinely conducted prior to clinical 
trials. This would provide robust, justifiable, empirical esti- 
mates of the SWE for use in sample size calculations and 
the interpretation of trial findings. It is possible that, if 
these simple procedures were widely adopted, clinical trials 
could be very different in size; some clinical conditions 
might be managed very differently, and health resources 
might be radically redistributed. 
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